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Method and Apparatus for improved pixel-based subtitling 



The invention relates to a method and to an apparatus for 
s improved pixel-based subtitling, which can be used e.g. for 
HDTV subtitles in pre-recorded formats like the so-called 
Blue-ray Disc. 



10 Background 

The technique of subtitling for Audio-Visual material has 
been used beginning with the first celluloid cinema movies 
and further until the recent digital media appeared. The 
15 main target of subtitling has been the support of handi- 
capped people or small ethnographic language groups. There- 
fore subtitling often aims at the presentation of text in- 
formation even when having been encoded as graphic data like 
pixel maps. Therefore pre-produced AV-material for broad- 
casting (Closed Caption, Teletext, DVB-Subtitle etc.) and 
movie discs (DVD Sub-Picture etc.) primarily are optimized 
for subtitles representing simple static textual informa- 
tion. However, progress in PC software development- for 
presentation and animation of textual information induces a 
corresponding demand for possibilities and features within 
the digital subtitling technique used for pre-recording and 
broadcasting. Using straightforward approaches without any 
special precautions, these increased requirements for subti- 
tling would consume a too big portion of the limited overall 
30 bandwidth. The conflicting requirements for a 'full fea- 
ture' subtitle encompassing karaoke all through genuine an- 
imations are on one hand the coding efficiency and on the 
other hand the full control for any subtitle author. 



20 



25 



35 



For today's state of the art of digitally subtitling AV ma- 
terial with separate subtitling information two main ap- 
proaches exist : Subtitling can be based on either pixel 



i » . r rt 



NOV. 2002 11:59 

PD02010 



m- 



1S1102 



NR. 753 



S.9/26 



data or on character data. In both cases, subtitling 
schemes comprise a general framework, which for instance 
deals with the synchronization of subtitling elements along 
the AV time axis. 

Character data based subtitling: 

In the character-based subtitling approach, e.g. in the 
teletext system [1] of European analog or digital TV, 
strings are described by sequences of letter codes (e.g. 

10 ASCII [2] or UNICODE [3] ) , which intrinsically allows for a 
very efficient encoding. But from character strings alone/ - 
subtitling can not be converted into a graphical representa- 
tion to be overlaid over video. For this, the intended 
character set, font and some font parameters, most notably 

15 the font si2e, must either be coded explicitly within the 
subtitling bitstream or an implicit assumption must .be made 
about them within a suitably defined subtitling context. 
Also, any subtitling in this approach is confined to what 
can be expressed with the letters and symbols of the spe- 

20 cific font(s) in use. The DVB Subtitling specification [4], 
in its mode of * character objects'' f constitutes another 
state-of-the-art example of character -based subtitling. 

Pixel data based subtitling: 

2s In the pixel -based subtitling approach, subtitling frames 
are conveyed directly in the form of graphical representa- 
tions by describing them as (typically rectangular) regions 
of pixel values on the AV screen. Whenever anything is 
meant to be visible in the subtitling plane superimposed - 

30 onto video, its pixel values must be encoded and provided in 
the subtitling bitstream, together with appropriate synchro- 
nization info and hence for the full feature animation of 
• subtitles all pixel changed must be transported. Obviously 
when removing any limitations inherent with full feature an- 

35 imations of teletext,, the pixel-based approach carries the 
penalty of a considerably increased bandwidth for the subti- 
tling data. Examples of pixel -based subtitling schemes can 
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be found in DVD's "Sub-picture" concept [5] as well as in 
the ""pixel object" concept of DVB Subtitling [4] . 

5 Invention 

The gist of the invention is a subtitling format encompass- 
ing elements of enhanced syntax and semantic to provide im- 
proved animation capabilities. The disclosed elements im- 

10 prove subtitle performance without stressing the available 
subtitle bitrate. This will become essential for authoring 
content of high-end HDTV subtitles in pre-recorded format 
which can be broadcast or pressed on high capacity optical 
media, e.g. the Blue-ray Disc. The invention includes 

X5 abilities for improve authoring possibilities for the con- 
tent production to animate subtitles. 

Introduced by the disclosure are elements of syntax and se- 
mantic describing the color change for parts of graphics to 
display. This can be used for highlight effects in appli- 
20 cations like for example karaoke, avoiding the repeated 
transfer of pixel data. 

Other disclosed elements of syntax and semantic facilitate 
the ability of cropping parts of the subtitles before dis- 
playing them. By using the technique of subsequent trans - 
25 f erred cropping parameters for an object to display a bit 
saving animation of subtitles becomes available. Such crop- 
ping parameter can be used for example to generate text 
changes by wiping boxes, blinds, scrolling, wipes, checker 
boxes, etc. 

30 Furthermore the disclosed elements can be used to provide 
interactivity on textual and graphical information. Espe- 
cially the positioning and/or color settings of subtitles 
can be manipulated based upon user request. 



35 

Drawings 
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Exemplary embodiments of the invention are described with 
reference to the accompanying drawings and tables, which 
show: 

5 Figure 1.- Example for the definition of a subtitle region 
and its location within a page; 

Figure 2: Example for definition of a region • sub -CLOT and 
region cropping; 

Figure 3: Resulting display example ; 
10 Figure 4: Interactive usage of subtitles ; 

Table 1: segment_type values for. enhanced. PCS . and. RCS ? 

Table 2 : Enhanced page composition segment / 

Table 3: Enhanced region composition segment; 

Figure 5: Video and Graphics Planes ; 
is Figure 6: Video and Graphics Mixing and Switching. 



Exemplary embodiments 

20 The invention can preferably be embodied based on the syntax 
and semantic of the DVB subtitle specification (DVB-ST) [4] . 
To provide improved capabilities for the manipulation of 
graphic subtitle elements the semantics of DVB-ST's page 
composition segment (PCS) and region composition seg- 

25 ment(RCS) are expanded. 

DVB_ST uses page composition segments (PCS) . to describe the 
positions of one or more rectangular regions on the display 
screen. The region composition segments (RCS) are used to 
define the size of any such rectangular area and identifies 

30 the color- lookup- table used within. 

The proposed invention keeps backward compatibility with 
DVB-ST by using different segment_typea for the enhanced PCS 
and RCS elements as listed in Table 1. It would of course 
be possible to choose other values instead. . Another ap- 

35 proach for keeping backward compatibility would be to keep 
the existing segment_types and increase the version_number 
of the specification (e.g. by incrementing the subti- 
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tle_stream_id in the PES_data_ r f ield structure) . The en- 
hanced PCS is defined as listed in Table 2 and the enhanced 
RCS is defined as listed in Table 3. with respect to origi- 
nal dvb-st, all structures shown are expanded, in the tables 
s the additional entries are marked gray. 

The enhanced PCS carries optional information about the re- 
gion cropping and optional information about the region sub- 
CLUT for every region listed. The 2 values of re- 
gioncropping and region^subaUT indicate if such optional 
io information is available for the current region in process 
Therefore cropping and sub-CLUT may be defined separately 
for every region. While region_cropping is, used as a flag 
(if equal 0x01) , the region_sub_CLUT shows the value how 
many sub-CLUT positions are described. This is done to pro- 
is vide different alternatives within the stream. Alternative 
sub-CLUT positions can be used to define different menu but- 
ton positions for the display screen, only one of them - 
the first one as a default - is active and the user can 
change the position to navigate through the different prede- 
20 fined positions pressing the remote for example. 

The enhanced RCS carries the sub_CLUT_id identifying the 
family of CLUTs that applies to this region. This is done 
to re-use CLUTs for different regions and different region 
sub_CLUTs as well. 

25 The enhanced PCS and enhanced RCS elements provide the abil- 
ity that subtitles can be manipulated independent from the 
encoding method i.e. independent from whether they are en- 
coded as character data or pixel data. 

The enhanced PCS and RCS can be used to perform many dif f er- 
30 ent animation effects for subtitles. Those could be wiping 
boxes, blinds,. scrolling, wipes, checker boxes, etc. The 
following Figures show an application example for karaoke 
• Figure l shows the definition of a region containing lyrics 

as tL a m r n b displ r d for karaoke - The iett «~ ° f 

35 tie may be encoded as pixel data or as character data as 
well. The regionvertical address and re- 

gion_hori 2 ontal_address define the location of the subtitle 
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within the frame to display. 

Figure 2 depicts the region cropping and the location of the 
region sub-CLUT, 

The region cropping defines which part of the region is ef- 
5 fectively displayed. This is achieved by 4 parameters indi- 
cating the start coordinates and the size of the fragment to 
display. region_horizontal_cropping specifies the horizon- 
tal address of the top left pixel of this cropping and re- 
gion_vertical_cropping specifies the vertical address of the 
10 top line of this cropping, region_cropping_width specifies 
. ...the horizontal length of this cropping and re- 

gion_cropping_height specifies the vertical length of this 
cropping . 

The region sub-CLUT location defines which part of the re- 

15 gion has to be displayed using a color look up table differ- 
ent from the region CLUT* This is achieved by 4 parameters 
indicating the start coordinates and the size of the sub- 
region used by the sub-CLUT. All coordinate parameters are 
to be understood relative to the region the sub-CLUT belongs 

20 to. sub_CLUT_horizontal_address specifies the horizontal 
address of the top left pixel of this sub-CLUT and 
sub_CLUT_vertical_address specifies the vertical address of 
the top line of this sub-CLUT. sub_CLUT_width specifies the 
horizontal length of this sub-CLUT and sub_CLUT_height 

25 specifies the vertical length of this sub-CLUT. 

Picking up all parameters defined with the previous Figures 
results in the displayed subtitle as depicted in Figure 3 . 
The subtitle is not depicted in whole but only a part of it. 
Furthermore the sub-CLUT was used to provide a highlight, so 

30 that the user knows what to sing in the moment. 

As the enhanced PCS are sent within MPEG PES packets labeled 
by presentation time stamps PTS any effect can be synchro- 
nized to the AV. 

Another idea of the invention is the superseding of subtitle 
35 animation parameters by the user. This offers a way to re- 
alize interactive subtitles. The enhanced PCS parameters 
are transferred as a default and the user may change them 
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via a remote control for example. By this the user is able 
to move, crop or highlight the subtitle. 

This could be an advantage for a user defined repositioning 
of a subtitling text, so that the user can subjectively 
minimize the annoyance by the subtitle text placement on top 
of the motion video. Also the color of the subtitles could 
be set according to users preferences. Figure 4 shows a 
block diagram for interactive subtitle modifications. The 
default parameters are superseded by the user action. 
Another application for overriding subtitle animation pa- 
rameters like position, cropping rectangle, cluts, sub-cluts 
is the realization of some very basic sort of interactive 
gaming. The subtitle may carry pixel data of an animated 
character. This character is subsequently moved on the dis- 
play screen driven by either user interaction, programmatic 
control or both. 

The overriding of subtitle animation parameters can be im- 
plemented in at least two ways. The first option is that the 
overriding parameters replace the parameters send in the 
bit stream. The second option is that the overriding parame- 
ters are used as an offset that is added to or subtracted 
from the subtitle animation parameters send in the bit- 
stream. 

The enhances PCS and RCS provide a lot more of animation ca- 
pabilities not explained. Following is a non-exhaustive list 
of exemples: wiping boxes, blinds, scrolling, wipes, checker 
boxes in details. 
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Claims 



!• Method for improved pixel-based subtitling, wherein 
subtitling frames are conveyed in* the form of graphical 
5 representations by describing them as regions of pixel 

values , wherein optional information about cropping and/or 
color change for parts of a region is provided. 



2. Apparatus for performing a method according to claim 1. 

10 
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Abstract 



The gist of the invention is a subtitling format encompass- 
ing elements of enhanced syntax and semantic to provide im- 
proved animation capabilities. The disclosed elements im- 
prove subtitle performance without stressing the available 
subtitle bitrate. This will become essential for authoring 
content of high-end HDTV subtitles in pre-recorded format 
which can be broadcast or pressed on high capacity optical 
media, e.g. the Blue-ray Disc. The invention includes 
abilities for improve authoring possibilities for . the. con- 
tent production to animate subtitles. 
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